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I.  Introduction 

Background  and  Purpose: 

Since  1978,  the  Health  Care  Financing  Administration  (HCFA),  now  CMS,  has  produced 
an  audio  edition  of  the  Medicare  handbook.  The  annual  total  of  recorded  handbooks  requested, 
from  1978  through  1998,  never  exceeded  1,500.  Thanks  to  timely  production  and  vigorous 
product  promotion  by  CMS's  Center  for  Beneficiary  Services  and  the  various  Medicare  outreach 
partners,  production  levels  to  meet  requests  for  the  recorded  edition  of  the  handbook  soared  to 
15,000  in  1999.  For  the  years  2000/2001,  requests  for  the  recorded  handbook  are  anticipated  to 
reach  an  all  time  high  of  30,000.  The  decision  to  consumer  test  the  audio  edition  of  the  Medicare 
handbook  came  as  a  result  of  the  product's  growing  popularity,  coupled  with  CMS's 
commitment  to  provide  blind,  low  vision  and  other  print  disabled  consumers  with  equal  and 
timely  access  to  high  quality  information  about  Medicare. 
Highlights  of  the  Study  Plan 

In  April  2001,  the  Policy  Research  and  Program  Evaluation  (PRPE)  Department  of  the 
American  Foundation  for  the  Blind  (AFB)  submitted  its  plan  to  conduct  a  mixed  methodology 
research  design  for  consumer  testing  the  audio  handbook.  This  included  one  telephone  focus 
group  session  with  individuals  across  the  country  who  are  experienced  audiotape  listeners, 
whether  or  not  they  had  an  association  with  Medicare,  and  8-10  in-person  observation/interview 
sessions  (based  on  a  "usability"  model)  with  blind/visually  impaired  individuals,  who  differed  in 
the  reasons  they  would  consult  the  audiotape  version  of  the  Medicare  and  You  2001.   These  blind 
or  low  vision  participants  were  either  a)  Medicare-entitled  beneficiaries  between  the  ages  of  65 


and  75,  b)  disabled  beneficiaries,  entitled  to  Medicare  for  at  least  12  months,  between  the  ages  21 
and  65,  and  c)  blind  or  low  vision  individuals  who  provide  care  or  health  care  advice  for 
Medicare  beneficiaries. 

To  recruit  beneficiaries,  a  "snowball"  referral  strategy  was  used:  private  blindness 
rehabilitation  agencies  were  contacted  and  asked  to  recommend  possible  participants.  AFB  then 
proceeded  to  contact  and  screen  individuals  for  eligibility  according  to  the  above  categories, 
making  specific  arrangements  to  respect  individuals'  privacy  rights.  Further  details  about 
recruitment  are  provided  in  the  methodology  section  attached  at  back. 

AFB's  research  team  participated  in  planning  with  CMS's  Project  Officer  by  telephone 
and  email.  The  focus  group  was  conducted  from  New  York  headquarters  using  a  teleconference 
service.  The  in-person  interviews  were  conducted  in  individuals'  homes  or  offices  in  New  York 
City  and  the  outlying  metropolitan  area  in  New  Jersey.    After  each  session,  AFB  provided  notes 
about  salient  points  of  procedure  or  content  as  well  as  an  audiotape  of  the  sessions,  to  share  in 
an  informal  debriefing  with  CMS's  Project  Manager,  by  phone  and  email.   Finally,  AFB 
analyzed  the  data  and  provided  a  "top-line"  summary  report  by  telephone  (June  27,  2001). 
This  final  report  presents  more  detailed  findings  along  with  recommendations  geared  toward 
revising  the  audio  edition  of  the  Medicare  and  You  handbook  for  the  2002  edition. 

The  next  section  reviews  prior  research  on  formatting  reference  guides  for  audiotape, 
and  raises  the  fundamental  question:  Do  recorded  publications  need  a  different  style  of 
information  organization  from  that  of  the  print  edition  in  order  to  be  an  effective  reference 
tool?  Some  of  the  difficulties  accessing  information  auditorily  are  discussed. 


II.   Perspective  on  Formatting  Reference  Materials  for  Audiotape 

While  there  are  guidelines  established  for  formatting  audio  materials  (see,  for  example. 
National  Braille  Association's  [NBA]  "Tape  Recording  Manual",  or  National  Library  Service  for 
the  Blind  and  Physically  Handicapped's  [NLS]  "Book  Mastering"),  we  know  of  no  previous 
research  which  determined  how  these  guidelines  were  established  or  which  evaluates  whether  the 
standards  that  exist  are  the  best  possible  design(s)  for  producing  reference  works  on  audiotape. 
from  a  user's  perspective  (American  Printing  House  [APH],  National  Library  Service  [NLS],  and 
Recording  for  the  Blind  and  Dyslexic  [RFB&D],  personal  communication). 

Many  of  the  specifications  and  recommendations  in  those  guidelines  reflect  a 
philosophical  commitment  to  ensure  equal  access  to  the  printed  word  for  individuals,  who 
because  of  visual,  learning  or  physical  impairments,  cannot  access  conventional  print  media. 
Traditionally,  this  commitment  has  meant  an  effort  to  ensure  that  audio  recordings  conform  to 
the  print  editions  in  their  entirety,  including  bibliographies,  appendices,  and  notes.  Further,  it  has 
meant  that  for  items  that  cannot  be  "read"  (such  as  illustrations  and  diagrams),  supplemental 
sections  are  written  so  that  items  are  "described"  to  the  reader  (NLS,  2000;  NBA,  1996).  While 
verbatim  narrations  function  very  well  for  works  of  fiction  or  other  genres  intended  to  be 
experienced  in  a  linear  fashion,  from  start  to  finish,  it  is  questionable  how  well  these  guidelines 
serve  other  formats.  The  possibility  that  recorded  publications  need  a  different  style  of 
information  organization  from  that  of  the  print  edition  in  order  to  be  used  as  an  effective 
reference  tool  came  up  recurrently  during  this  research. 

In  order  to  effectively  use  the  audio  edition  of  the  Medicare  and  You  handbook,  as  well 


as  other  reference  materials,  consumers  are  required  to  manipulate  the  text  in  a  non-linear 
fashion--  e.g.,  to  make  use  of  the  glossary  in  the  middle  of  their  reading;  to  find  a  particular 
phone  number  appropriate  to  their  State;  or  to  listen  just  to  the  section  dealing  with  hospice  stays 
or  reimbursement  for  medications  ~  whatever  it  is  that  suits  the  needs  of  the  consumer  at  that 
particular  moment.  However,  "jumping  around"  through  a  document,  "skipping  ahead",  or 
"flipping  back  and  forth"  are  actions  print  users  take;  these  tasks  are  difficult  for  users  of 
audiotapes,  given  the  nature  of  the  medium. 

Tone  indexing  has  been  the  industry  solution  to  deal  with  these  navigational  barriers  for 
tape  users,  and  many  of  the  guidelines  mentioned  above  reflect  advances  in  recording  techniques 
to  better  accommodate  these  needs.  However,  the  guidelines  do  not  reflect  how  users  actually 
use  such  references.  Indeed,  many  of  the  recommendations  presented  below  reflect  design 
features  that  accommodate  users'  needs,  rather  than  technological  sophistication.  Moreover,  it 
has  been  argued  that  eventually  this  question  will  become  moot  as  digital  recordings  and  digital 
talking  books  (DTB)  replace  tape  as  the  primary  medium  for  non-braille,  print  disabled  readers. 
We  argue,  however,  that  it  will  be  a  long  time  before  DTBs  become  standard  in  every  household 
and  that  there  is  therefore  a  need  for  further  research  of  this  type.  Consequently,  we  have 
included  in  our  findings  recommendations  specific  to  the  Medicare  and  You  handbook,  as  well  as 
some  that  have  more  generalizable  appeal. 


III.  Findings  and  Recommendations 

A.  About  the  Participants 

There  were  14  participants  in  total:  5  individuals  in  the  telephone  focus  group  and  9 
individuals  with  whom  we  conducted  in-person  interviews/observation.  Because  the  focus  group 
was  conducted  by  telephone,  geographic  diversity  was  achieved;  all  four  Census  regions  were 
represented.  The  in-person  "usability"  component  was  not  geographically  diverse,  due  to 
budgetary  and  time  constraints.  However,  these  individuals  did  vary  according  to  their  Medicare 
needs  (their  relationship  to  Medicare:  "a",  "b"  or  "c"  categories,  mentioned  above),  their 
experience  as  an  auditory  reader,  age,  and  gender.  Please  see  also  Section  V.  Methodology  for 
further  details. 

Note:  about  the  findings  and  recommendations  below.  First,  the  points  below  are  not  presented 
in  order  of  importance.  Second,  the  findings  and  recommendations  below  are  presented  together. 
Given  the  nature  of  the  content,  we  felt  it  was  more  convenient  for  the  reader  to  hear  the 
recommendations  directly  after  each  point  is  explained.  Third,  this  project  did  not  have  budget 
for  transcription  of  the  taped  sessions;  I  have  included  some  direct  quotes,  nonetheless,  when 
possible.  Participants'  comments  are  also  included  indirectly  with  their  first  name  and  ID 
number  following  in  parenthesis.  Generally  speaking,  participants  felt  that  the  Medicare  and 
You  2001  Handbook,  audio  edition  would  suffice  as  a  basic  reference  document,  because  it  was 
important  to  have  accessible  information,  but  not  necessarily  because  they  were  satisfied  with  the 
format  as  it  currently  stands.  Their  specific  suggestions  for  improving  this  reference  guide  are 
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indicated  below. 

B.  Findings  about  Access 

1.  More  promotion  of  alternate  formats  needed. 

Only  two  of  all  of  the  research  participants,  including  those  who  are  professional 
advisors  on  health  care  knew  that  Medicare  had  materials  in  alternate  formats  and  that  the 
audio  handbook  existed  prior  to  our  contacting  them.   They  agreed  that  by  far  the  most 
common  way  to  get  information  about  Medicare  changes  was  by  word  of  mouth  (friends, 
colleagues,  doctors,  etc).   They  recommended  more  advertising  through  sister  agencies  or 
programs  (such  as  the  Social  Security  Administration,  Medicaid,  etc),  and  putting  notification 
on  the  front  cover  of  print  materials.   Additionally,  they  identified  the  following  to  increase 
promotion:   agencies  for  the  blind;  organizations  of,  and  activities  favored  by,  visually 
impaired  consumer  groups;  announcements  to  listservs;  speakers  at  Senior  Centers;  and  public 
service  announcements  (PSAs)  on  radio  and  TV.   The  best  way  to  get  updated  information  to 
consumers  would  be  to  send  them  something  directly  (e.g.,  a  letter  on  tape  or  in  braille),  rather 
than  putting  the  onus  on  them  to  seek  it  out. 

2.  Packaging. 

The  external  packaging  was  difficult  for  some  individuals  to  figure  out  where  to  open. 
Additionally,  several  users  had  difficulty,  physically,  opening  the  package,  and/or  pulling  out 
the  tapes.   A  few  felt  the  packaging  was  sharp  at  the  edges,  and  that  it  might  be  difficult  for 
beneficiaries  with  weakness  or  arthritis  to  manipulate.   Although  they  did  not  specify  a 


particular  alternative,  they  recommended  packaging  that  is  easier  to  open. 

It  was  not  clear  for  non-braille  readers  which  tape  was  Side  1;  the  braille  is  too  small 
and  low  for  elderly  persons  to  read  clearly.   They  suggested  larger,  more  pronounced  braille, 
and  indication  of  side  1  and  2,  etc.,  as  is  done  on  standard  cassettes,  or  use  of  large  raised 
numbers. 

3.   More  introductory  information  for  beginner  users. 

Participants  requested  more  introductory  material  for  beginner  tape  users;  they 
universally  indicated  that  this  would  help  aid  navigation.   This  introductory  material  should 
include:   a)  description  of  "the  bulk"  of  the  volume;  b)  a  sample  of  a  tone;  (they  suggested 
considering  calling  the  "tone  indexing"  "beeps")  and,  c)  explaining  how  the  use  of  these  tapes 
differs  from  four-track  NLS  tapes.   (Sara#7).   There  was  some  question  as  to  whether  the 
supplemental  introductory  material  should  be  up-front  or  included  as  a  separate  tape. 

C.  Findings  about  Navigation 

In  general,  individuals  felt  this  document  was  difficult  to  navigate.  Almost  all  had 
problems  hearing  the  tones,  and  hence  navigating  by  them.   They  generally  agreed  it  would 
suffice  as  a  basic  reference  document,  but  felt  they  would  probably  have  to  listen  to  it  all  the 
way  through  in  order  to  get  what  they  needed,  even  though  they  much  preferred  being  able  to 
jump  to  just  whichever  sections  were  relevant  to  them.   We  suggest  that  the  reason  they  said  it 

10 


"sufficed"  reflected  appreciation  for  having  accessible  materials  available  to  them,  more  than  it 
reflected  satisfaction  that  this  was  in  a  workable  form.  Details  for  improving  the  tone  indexing, 
as  well  as  other  navigational  tips  are  included  below. 

1.  Specific  problems  with  the  tone  indexing. 

A  lot  of  suggestions  for  improving  the  tone  indexing  were  mentioned.   These  include: 
a)  including  an  example  of  various  tones  to  explain  tone  indexing,  b)  more  tone  indexing 
(sections  were  very  long  with  no  tones),  c)  experimenting  with  different  tones  to  convey 
different  things,  e.g.,  double  tones  for  major  sections  and  single  tones  for  secondary  elements 
(or  longer  and  shorter  ones),  and  d)  longer  spaces  and  more  pausing  between  elements.   They 
wanted  to  be  able  to  hear  the  tones  in  fast  forward,  play,  and  rewind  mode,  and  weren't  always 
able  to  do  so.   While  the  quality  of  the  tones  varied  from  machine  to  machine,  they  felt  it  was 
important  to  adhere  to  the  specific  LOC  standards  (5  seconds,  60KHZ)  in  addition  to  any  other 
changes  which  might  be  made. 

2.  Confusing  reference  markers. 

Specifically,  a)  change  "section  3"  to  "Side  3"  (people  thought  "section"  and  "side" 
were  synonymous),  and  include  on  which  side  of  the  tapes  the  tones  were  located  (e.g.,  even  if 
they  weren't  confused  about  which  section,  they  wouldn't  know  which  side  of  the  tape  to 
play),  b)  it  would  be  easier  for  listeners  if  the  number  of  the  tones  started  a  new  for  each  tape 
side,  e.g.,  change  "Tones  7-8"  to  "tones  1  and  2  on  side  3".   Additionally,  they  wanted  the 
information  helpful  to  tape  listeners  to  be  read  first.   For  example,  "Side  3,  tone  4,  that  will  be 
Section  6,  pp  22-23." 
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3.  Accommodate  different  navigational  approaches. 

There  were  several  different  ways  individuals  navigated  through  the  document,  each  of 
which  has  implications  for  formatting  reference  documents:   a)  "classic"  style  -  e.g.,  listening 
for  tone  5,  and  counting  the  number  of  beeps  until  they  get  to  what  they  were  looking  for,  b) 
listening  for  and  stopping  after  each  tone,  to  hear  where  they  were,  rather  than  counting  and 
remembering  which  tone  they  were  at  (e.g.,  tone  8)  and,  c)  listening  to  entire  sections,  but  at 
an  accelerated  speed.  Consequently,  the  narrator  should  announce  where  they  are  after  each 
tone,  what  the  name  of  that  section  is,  and  tones  should  be  clearly  loud  enough  (piercing)  in 
play  mode  as  well  as  fast  forward. 

4.  Page  numbers. 

A  question  was  raised  during  the  focus  group  about  the  necessity  of  starting  page 
numbers  (because  the  "references"  were  cumbersome).   However,  it  is  important  to  include 
page  numbers,  as  some  readers  will  use  them  to  navigate  (i.e.,  because  the  page  numbers  are 
the  most  frequently  spoken  navigational  aid,  when  users  stop  at  random  after  fast  forwarding 
the  tape,  they  can  hear  if  they  have  gone  far  enough  or  too  far).   Page  numbers  also  enable 
individuals  to  compare  information  with  sighted  family  members,  Medicare  employees,  and 
case  workers. 

5.  Pauses. 

It  would  be  helpful  to  insert  more  pauses  between  items  in  a  list  (e.g.,  since  it  is  not 
possible  to  tone  index  each  State's  phone  number)  and  into  really  long  sections  (e.g., "section 
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3"),  so  users  could  differentiate  between  different  parts  of  that  section.   This  would  enable 
users  to  navigate  by  breaks. 

6.  Alphabetical  and  numeric  listings. 

Individuals  preferred  to  have  any  compilation  of  information  listed  alphabetically. 
Further,  they  assumed  that  the  listings  would  be  structured  this  way  (i.e.,  State  phone  numbers) 
and  used  this  assumption  to  orient  themselves  and  to  navigate.  However,  because  the  "phone 
number"  section  was  organized  by  topic  (i.e.  Regional  Home  Health  Intermediary,  then  Peer 
Review  Organization,  then  State  Insurance  Department),  users  didn't  know  where  in  the  set  of 
listings  they  were  when  they  heard  "New  Jersey".  It  would  be  more  useful  to  list  phone  numbers 
by  State.  Topics  should  then  be  listed  alphabetically  within  each  State. 

7.  Avoid  Sidebars  and  footnotes  which  are  disruptive. 

For  example,  there  is  a  sidebar  about  "California".   One  user  was  trying  to  find 
information  about  New  York.   Because  this  individual  was  navigating  within  the  phone 
numbers  by  fast  forwarding  for  a  random  length,  then  hitting  play  for  a  split  second  to  hear 
where  he  was,  he  heard  "California"  and  thought  he  was  in  the  wrong  place. 

Sidebars  and  footnotes  also  contribute  to  "audio  overload"  (e.g.,  pp  10  and  11). 
Sidebar  and  footnote  information  (indicated  with  an  asterisk  in  the  print  version)  should  be 
read  only  once,  with  an  explanatory  sentence,  if  need  be,  to  indicate  that  the  information 
applies  to  the  entire  section. 
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8.   Include  more  "text  description". 

It  would  help  auditory  readers  to  hear  things  such  as  "This  is  the  top  of  a  chart, 
entitled..."   And,  "End  of  chart."   For  the  most  part,  charts  are  a  visual  and  not  an  auditory 
aid,  and  the  format,  while  helpful  for  sighted  readers,  can  be  confusing  for  tape  readers. 
Verbal  clues  as  to  the  design  and  layout  of  the  formatting,  therefore,  are  helpful  for  orientation 
and  for  navigation.   Similarly,  almost  all  listeners  felt  they  wanted  a  more  explicit  "Table  of 
Contents"  and  wanted  it  to  come  earlier  in  the  document.   They  also  wanted  the  "mini"  Table 
of  Contents  that  is  at  the  beginning  of  each  side  to  be  more  explicit.   There  was  one  point  (see 
Maria#2,  Eddie#4)  where  it  sounded  like  the  narrator  was  reading  different  page  numbers  for 
the  same  section.   The  confusion  resulted  because  individuals  did  not  know  they  were  in  a 
chart.   It  should  have  read,  "End  of  the  Table  of  Contents.   We  are  beginning  now  with 
manual,  page  __  which  has  a  chart,  entitled ". 

D.  Findings  about  Content 

1 .   Location  of  Errata  sheet. 

Although  digital  formats  in  the  future  may  make  this  point  moot,  it  is  worth  mentioning 
that  there  was  confusion  caused  by  the  location  of  the  Errata  sheet  in  this  version  of  the 
handbook.   This  was  a  consistent  finding  that  affected  orientation  as  well  as  comprehension. 
For  example,  the  tape  does  not  explain,  "beginning  of  the  errata  sheet",  or  "we  are  now  going 
to  read  a  list  of  changes  that  has  been  inserted  out  of  order  in  the  handbook",  etc.  Therefore. 


14 


people  were  confused  about  content,  because  it  did  not  give  enough  detail.    For  example,  the 
"Errata"  sheet  mentions  a  change  relating  to  the  care  beginning  with  the  21st  day.    One 
participant  asked  (Ed#6),  "What  happens  for  the  first  20  days?   She  talks  about  hospital  stays 
then  skilled  nursing  facilities.  She  does  not  give  all  the  information."   Another  tester  explained 
that  it's  "out  of  context"  to  have  the  changes  in  the  beginning/introduction,  not  because  they 
are  in  the  beginning,  but  because  they  don't  have  further  explanation  (see  Sara#7).    Instead, 
they  suggested  including  an  explanatory  sentence,  such  as  "The  following  items  will  be 
explained  in  greater  detail  later;  we  are  just  announcing  any  changes  below..."  (They  even 
suggested  modeling  it  on  the  familiar  case  of  "headlines,  news  at  11").  Content  would  be 
clearer  if  the  Table  of  Contents  was  earlier  (see  above),  and  the  glossary  was  up  front  as  well. 

2.   Jargon 

Participants  felt  that  there  were  cases  when  "jargony"  language  was  used,  and  that  these 
technical  terms  were  used  too  early  in  the  document  without  being  defined.   For  example,  a) 
One  user  asked,  "What  is  a  'benefit  period'?  No  one  is  going  to  know  what  that  is.   It  jumps 
too  quickly  into  the  detail  without  defining  the  term."   b)  "skilled  nursing  facility"  also 
presented  problems;  someone  not  acquainted  with  this  term  does  not  know  what  it  is. 
Furthermore,  several  individuals,  including  two  social  workers,  suggested  others  would  be 
scared  by  the  content  of  the  Errata  sheet  so  early  on;  they  would  think,  for  example,  that  they 
have  to  pay  a  lot  of  money,  and  would  not  continue  listening  to  the  rest  of  the  tape,  c)  "No 
change  in  Part  B  deductible"  should  be  followed  by  a  sentence  that  says,  "  The  deductible  is 
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the  amount  you  pay..."     By  contrast,  "You  may  be  able  to  get  help  paying..."  was  good;  the 
language  was  clear  and  not  jargony.   It  is  explicit. 

Additionally,  we  have  reason  to  believe  (from  the  interviews  with  service  workers)  that 
individuals  confused  Medicare  with  Medicaid,  and  a  lot  of  people  don't  know  what  Medicare 
Part  B  is.   Put  "What  is  Part  B?"  in  Part  B  section.   (See  also  Beverly#5). 

3.  Include  "content"  about  how  to  use  Medicare. 

One  social  service  worker  told  us  that  her  clientele  are  confused  by  receiving  something 
in  the  mail  entitled  "This  is  not  a  bill."  She  suggested  putting  this  information  (as  well  as  other 
information  about  how  to  use  the  system)  into  the  audio  handbook.   For  example,  "when  you 
use  your  Medicare  card,  you  will  receive  a  statement  that  is  not  a  bill." 

4.  Issue  of  "length  of  tapes". 

It  was  suggested  by  a  category  "c"  participant  who  works  with  elderly  and  chronically 
ill  clients  that  much  shorter  tapes  are  needed  for  beneficiaries  who  are  ill.   They  don't  have 
enough  time  when  they  are  feeling  well,  or  the  attention  span,  to  listen  to  such  lengthy 
materials.  (See  Mindy  #9). 

5.  Narrator's  voice. 

All  users  felt  the  narrator's  voice  was  clear  and  understandable.   A  few  felt  it  was  a 
little  sing-songy,  but  acceptable  (i.e.,  as  one  user  said,  "How  interesting  can  you  make  a 
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Medicare  handbook?!   Just  read  the  darn  thing.")  There  was  no  preference  for  a  male  or 
female  voice.   However,  almost  all  users  felt  that  the  phone  numbers  should  be  read  more 
slowly  and  repeated.   They  suggested  pauses  between  each  set  of  digits  (e.g.,  "1-800"  pause 
"633"  pause,  etc).   (Additionally,  they  felt  the  "1-800-Medicare"  was  unnecessary;  blind  users 
preferred  the  actual  numbers).   The  references  ("section  3,  tone  4,  pp  15-27")  should  also  be 
read  more  slowly.   (They  felt  the  narrator's  tone  dropped  slightly  when  reading  the  references 
and  the  "go  to"  was  unnecessary.)  The  web  address  should  be  read  as  "www. medicare. g-o-v" 
instead  of  "gov". 

E.   Additional  Recommendations 

There  were  additional  recommendations  that  were  suggested  by  the  data  which  tended 
to  fall  beyond  the  scope  of  traditional  revisions  to  the  given  tape  format.   We  felt  it  was 
important  to  include  these  "avant  garde"  or  innovative  recommendations  because  we  did  not 
want  to  limit  the  scope  of  thinking.   Importantly,  we  believe  they  may  enhance  the  integrity  of 
the  handbook;  it  is  "thinking  out  of  the  box"  which  may  actually  produce  the  most  usable 
reference  guide  for  auditory  readers. 

As  mentioned  above,  this  research  raises  the  question.  Do  recorded  publications  need  a 
different  style  of  information  organization  from  that  of  the  print  edition  in  order  to  be  an 
effective  reference  tool?  There  were  many  examples  in  the  findings  where  being  not  so  closely 
wedded  to  the  print  version  might  actually  enhance  the  quality  of  the  book.   For  example. 
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hearing  the  Medicare  number  on  every  page  contributed  to  "audio  overload".   Listeners  felt 
that  if  the  numbers  were  clearly  re-iterated  at  the  beginning  of  the  tape  (and  perhaps  the 
beginning  of  each  side)  or  somewhere  else  easily  findable,  such  as  on  a  separate  tape,  then  one 
could  just  go  to  the  beginning  of  the  tape,  hear  the  number,  and  call  as  needed.   Similarly, 
moving  the  names  of  CMS  administrators  to  the  back,  and  other  navigational  aids  to  the  front 
(see  above)  would  be  more  helpful,  and  could  be  the  difference  between  using  or  not  using  the 
tapes.   Several  testers  stated  that  they  "would  have  given  up  after  a  few  minutes",  that  there 
was  "too  much  flipping  back  and  forth"  to  find  what  they  needed  and  "would  have  gone 
elsewhere;  this  format  wasn't  worth  the  time."   In  general,  anything  that  reduces  the  amount  of 
flipping  back  and  forth  (which  is  a  sighted  activity  not  listening  activity)  would  make  the 
reference  far  more  usable. 

Along  these  lines,  testers  recommended: 

a)  pulling  all  the  charts  out  from  the  body  of  the  document  and  including  them  at  the  back  for 

those  who  need  to  reference  the  printed  version,  with  a  note  stating,  "these  reference  pages 

to in  the  print  handbook. " 

b)  create  a  separate  tape  for  every  State,  and  include  only  the  information  relevant  to  that  State. 
Short  of  that,  organize  the  listings  alphabetically  by  State,  then  subdivide  within  each  State, 
rather  than  by  topic  (as  it  currently  is,  see  pp  25A-44). 
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c)  produce  several  different  versions  of  the  handbook.    Specifically,  make  one  that  is 
comprehensive  and  compares  verbatim  to  the  print  edition,  and  one  that  would  be  designed 
from  the  ground  up  for  listeners  of  that  material.   This  would  include  a  very  short,  overall 
and/or  introductory  tape,  shorter  tapes  in  general,  and  might  even  have  the  material  organized 
by  health  care  issue  or  need. 

d)  re-write  the  print  document  so  that  it  can  be  narrated  effectively. 

e)  create  a  registry  when  one  enrolls  in  or  becomes  eligible  for  Medicare,  and  use  it  to  send 
individuals  the  materials  in  their  preferred  format  each  year,  much  the  way  beneficiaries 
receive  the  updated  printed  handbook  each  year. 

f)  emboss  the  cover  of  the  print  version  with  a  line  of  braille  (or  a  sticky  label)  indicating  that 
this  handbook  is  available  in  alternate  formats.   Even  if  someone  didn't  read  braille,  they  might 
realize  that  this  document  was  meant  for  them,  and  act  as  a  trigger  to  inquire  about  other 
formats.   It  would  enhance  promotion,  even  for  sighted  users. 

g)  consider  innovative  alternatives  to  tapes  altogether.   While  technology  may  be  moving 
towards  digital  equipment,  users,  and  especially  the  elder  beneficiaries,  reiterated  their  comfort 
with  the  telephone  interface.   Participants  recommended  a  telephone  reading  service,  which 
could  be  navigated  by  speaking  key  words  into  the  phone  (i.e.,  "If  you  need  information  about 
,  please  say  "home  health  aids"  or  "medications",  etc). 
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IV.  Future  Research 

There  are  several  directions  for  future  research  which  have  been  identified  by  this 
project.  First,  it  would  be  advantageous  to  expand  the  range  and  scope  of  participants  in  the 
usability  component  to  the  study.   As  described  elsewhere,  the  constraints  of  time  and  budget 
required  a  locally-based  sample.   However,  it  would  likely  yield  a  better  understanding  of  the 
range  of  needs  and  capabilities  of  Medicare  beneficiaries,  if  research  included  people  from 
more  rural  areas  as  well  as  an  ethnically  diverse  population,  as  it  would  more  accurately  reflect 
part  of  target  population  who  are  not  as  well-linked  into  the  blindness  system.   These 
individuals  may  have  different  needs,  and  furthermore  could  help  to  identify  ways  to  reach 
other  "hard  to  reach"  individuals  about  changes  in  their  health  coverage.   Similarly,  we 
recommend  expanding  the  sample  of  providers  who  serve  these  populations,  as  they  may  be 
better  able  to  identify  the  specific  concerns  of  their  local  clientele.   With  greater  lead  time,  it 
would  be  possible  to  work  with  AFB's  regional  offices  (in  Atlanta,  Dallas,  Chicago,  and  San 
Francisco)  to  gather  a  more  geographically  representative  sample.   Similarly,  populations  of 
non-visually  impaired  print  disabled  consumers  deserve  further  attention.   See  the  limitations 
section  below. 

Second,  the  question  of  how  usable  the  Medicare  and  You  document  is  that  we 
addressed  here  does  not  tell  us  whether  or  not  individuals  actually  use  the  handbook,  once  they 
have  received  it.   There  may  be  a  difference  in  what  they  are  able  to  do  (i.e.,  with  a  researcher 
paying  them  to  devote  two  hours  to  this  concern)  versus  what  they  are  likely  to  do  when  faced 
with  a  health  care  information  need  (e.g.,  use  the  phone  instead).   Research  using 
"subscription  lists"  of  actual  users  would  be  able  to  address  how  individuals  use  these 
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documents  in  practice. 

Third,  this  research  indicates  that  consumers  feel  strongly  that  the  burden  to  identify 
changes  in  their  health  coverage  should  not  fall  entirely  to  them,  as  it  does  now,  just  because 
they  are  blind.   While  sighted  beneficiaries  are  sent  an  updated  handbook  each  year,  blind 
individuals  are  forced  to  inquire  after  alternate  materials:   more  research  should  explore 
additional  avenues  of  promotion,  so  these  individuals  would  know  these  alternate  materials 
were  available  to  them,  and  innovative  ways  of  getting  the  health  care  information  itself  to 
visually  impaired  consumers.   Some  of  the  suggestions  made  by  respondents  of  this  study 
towards  this  end  (such  as  a  phone  reading  service)  could  be  tested. 
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V.  Methodology 

A.  Respondent  Categories 

CMS  requested  that  four  types  of  handbook  users  be  involved: 

1.  Medicare  entitled  beneficiaries,  ages  65  to  approximately  75  (category  "a") 

2.  Disabled  beneficiaries,  entitled  to  Medicare  for  at  least  12  months,  ages  21  up  to  65 

(category  "b") 

3.  Blind  or  low  vision  individuals  who  provide  care  or  health  care  advice  for  Medicare 
entitled  beneficiaries  (category  "c") 

4.  Members  of  national  advocacy  groups  for  the  Blind  (e.g.,  NFB,  ACB)  or  others  with 

strong  audio-listening  skills,  who  may  or  may  not  have  a  direct  relationship  with 
Medicare  entitled  beneficiaries 
Additionally,  participants  in  the  consumer  testing  of  the  recorded  Medicare  handbook  must  meet 
the  following  criteria: 

--  Be  certifiable  as  blind,  low  vision  or  reading  disabled 

--  Have  a  minimum  of  12  months1  experience  using  recorded  materials  such  as  books  or 
periodicals  provided  by  the  National  Library  Service;  recorded  textbooks,  reports  or 
newsletters. 
--  Represent  major  U.S.  regions; 

--  If  entitled  to  Medicare,  have  some  decision-making  responsibility  for  personal  care 
matters; 
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-  Not  be  a  resident  of  a  nursing  home; 

~  Not  be  enrolled  in  a  managed  care  plan; 

—  Agree  to  listen  to  the  audio  recording  of  the  2001  Medicare  handbook,  in  its  entirety  if 

possible,  but  at  least  to  cover  pre-determined  sections,  prior  to  the  telephone  interview 

session; 
~  Agree  to  be  available  for  either  a  telephone  or  in-person  interview  session 
~  Not  be  dually  entitled  to  Medicare  and  Medicaid 

B.  Candidate  Selection 

Potential  participants  were  identified  and  contacted  by  partnering  organizations,  for 
purposes  of  protecting  and  respecting  personal  privacy.  Once  individuals  had  agreed  to 
participate,  and  have  contact  information  about  them  released  to  AFB,  AFB  contacted  them  to 
see  if  they  were  eligible  according  to  the  above  screening  criteria.  See  Table  1  for  a  list  of 
partnering  organizations  and  names  of  associated  contacts.  CMS's  Project  Officer  and  AFB's 
Research  Director  also  suggested  contacts. 
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Table  1:   Partnering  Organizations  in  the  Recruitment  Process 


Organization/ Agency 

Contact  Name 

American  Printing  House  for  the  Blind 

Mary  T.  Terlau 

Blind  Industries  and  Services  of  Maryland 

Ellen  Ringline 

Catholic  Guild  for  the  Blind,  NY 

Theresa  Snyder 

Center  for  the  Visually  Impaired,  Atlanta, 
GA 

Nell  Robinson 

Jewish  Guild  for  the  Blind,  NY 

Alan  Morse,  Elaine  Yatzkin,  Audrey 
Shadling 

National  Federation  of  the  Blind 

Pat  Maurer,  Everlee  Hairston  (Trenton,  NJ 
Chapter),  Linda  Debardinez,  Joe  Ruffalo 
(Essex  County,  NJ  Chapter) 

Passaic  County  Board  of  Social  Services,  NJ 

Mark  Steven 

Perkins  School  for  the  Blind,  MA 

Kim  Charlson 

Puerto  Rican  Association,  NJ 

Marza  Rodriguez 

Social  Security  Administration 

can  not  disclose  participants  name 

VISIONS,  NY 

Nancy  Weber-Miller,  Betsy  Fabricant 

Visiting  Home  Maker  Services,  NJ 

Cheryl  Dayton 

Once  identified,  potential  candidates  were  screened  over  the  telephone.  The  screening 
process  took  approximately  5-10  minutes  and  had  three  purposes:   1 .  To  establish  that  the 
participant  was  a  member  of  one  of  the  target  audiences,  2.  To  determine  that  each  was  willing 
and  able  to  participate  in  the  focus  group  on  the  scheduled  date,  or  to  set  up  an  appointment  for  a 
home  visit,  3.  To  collect  some  demographic  information  in  an  attempt  to  attain  diversity  in  our 
sample. 
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A  core  screening  protocol  was  used  to  explain  the  purpose  of  the  research,  indicate  their 
role  in  the  consumer  test  (focus  group  participants  were  given  a  task  assignment  prior  to  the 
session),  and  to  clarify  compensation  for  their  efforts  (subjects  were  reimbursed  $75  for  their 
participation).  Additionally,  participants  received  a  written  confirmation  of  these  arrangements 
in  the  individual's  preferred  format. 

One  participant  recruited  for  the  focus  group  had  a  schedule  conflict  and  had  to 
withdraw.  For  the  usability  component,  many  of  the  potential  participants  who  were  identified 
were  disqualified  because  they  were  also  receiving  Medicaid  or  in  some  other  way  did  not  meet 
the  eligibility  requirements.  The  overall  number  needed  was  attained  with  fewer  calls  than 
anticipated,  once  potential  participants  had  been  identified;  we  were  seeking  only  3  participants 
in  each  category.  There  were  18  individuals  identified  who  were  eligible  to  participate  (8  in 
category  "a";  5  in  category  "b";  5  in  category  "c").  Once  eligibility  was  confirmed,  individuals 
were  selected  with  the  goal  of  gathering  a  gender-balanced,  diverse  sample.  For  example,  it  was 
not  difficult  to  identify  females  who  fit  the  criteria  for  category  "a"  (beneficiaries  65-75  years  of 
age),  but  males  were  more  difficult  to  identify,  so  the  one  male  who  responded  to  our  call  in  that 
category  was  chosen.  By  far,  the  most  difficult  category  to  recruit  were  the  caregivers  or  advice 
givers  who  are  blind/visually  impaired  (category  "c").  With  longer  lead  time,  we  could  have 
identified  more  people  in  this  category;  see  directions  for  future  research. 
See  Table  2. 
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Table  2:   Participant  Information  Involved  in  the  Consumer  Test 


Total  Number  of  Participants 

14 

Focus  Group 

5 

In  Person  Interview 

9 

Gender 

Number 

Focus  Group 

Male 

2 

Female 

3 

In  person  Interviews 

Male 

3 

Female 

6 

Participant  Category 

All  Focus  Group  Participants  were 
experienced  audio  listeners. 

In-  person  Interviews 

Total  Category  "A"1 

3 

Male 

1 

Female 

2 

Total  Category  "B" 

3 

Male 

2 

Female 

1 

Total  Category  "C" 

3 

Male 

0 

Female 

3 

'Category  "A"  represents  Medicare  beneficiaries,  ages  65-75;  Category  "B"  represents 
disabled  beneficiaries,  ages  21-65  (entitled  to  Medicare  for  at  least  12  months);  Category  "C" 
represents  blind  or  visually  impaired  individuals  who  provide  formal  or  informal  care  or  advice  to 
Medicare  beneficiaries. 
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C.   Focus  Group  and  Usability  Guides 

The  goal  of  both  the  focus  group  and  usability  data  collection  was  to  evaluate  current 
product  features  such  as  descriptions  of  charts  and  graphics;  structure  of  recorded  handbook 
elements;  tone  and  page  references;  quality  of  the  narrator's  voice;  and  overall  improvements  in 
how  consumers  use  and  understand  concepts  and  the  handbook. 

The  focus  group  guide  was  drafted  based  on  several  sources: 

a)  planning  meetings  with  CMS  staff  about  their  information  needs 

b)  previous  research  conducted  on  the  print  handbook,  adapted  for  the  specific  assessment 

needs  of  the  audio  version 

In  general,  the  format  was:  the  moderator/interviewer's  introduction,  followed  by  the 
participants'  introduction  with  response  to  an  "ice-breaker"  question  (i.e.,  "What  do  you  like  best 
about  reading  by  tape?")  Next,  topics  included  overall  impressions,  and  specific  questions  about 
both  content  and  navigability.  This  structure  was  altered  during  the  in-person  interviews  to  allow 
the  consumers  to  listen  to  the  entirety  of  the  introduction  before  continuing  to  probe  about 
content  questions.  It  was  from  these  content  questions  that  "tasks"  to  assess  navigation  were 
selected.  As  mentioned  above,  the  goal  was  to  choose  tasks  which  were  user  driven;  the 
assumption  is  the  more  interest  users  have,  the  more  motivated  they  will  be  to  accomplish  the 
assignment  successfully. 

Copies  of  both  guides  are  attached  at  back. 
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D.  Length  and  Recording 

Data  collection  was  conducted  over  a  four  week  period,  beginning  mid  May.  The  focus 
group  lasted  for  1.5  hours,  although  participants  in  effect  spent  a  great  deal  of  time  with  this 
project,  as  they  were  asked  to  read  the  entire  handbook  (3  hours),  if  possible,  and  complete  a 
"task  assignment"  prior  to  the  scheduled  phone  meeting.  We  requested  that  individuals 
participating  in  the  "usability"  component  of  the  study  allot  two  hours  for  the  interview,  which  is 
roughly  how  long  they  ran.  These  individuals  were  not  required  to  review  any  advance 
materials,  as  we  wanted  to  evaluate  their  initial  impressions  and  assessment  of  the  packaging  as 
well  as  the  content. 

E.  Limitations/Strengths 

a)  As  mentioned  above,  the  "usability"  data  were  limited  geographically,  due  to  time  and 
budgetary  constraints.  This  sample  was  neither  representative  nor  random,  and  we  feel  strongly 
that  further  research  should  attempt  to  find  subjects  from  a  more  diverse  populace,  who  may  be 
less  tied-in  to  blindness  services  and  agencies,  as  was  our  sample  from  New  York,  and  who  may 
be  less  well  versed  in  manipulating  taped  recordings.  Additionally,  in  the  sample  above,  we 
limited  participants  to  those  who  were  not  receiving  Medicaid  in  order  to  control  for  economic 
differences.  Further,  there  were  no  non-visually  impaired  print-disabled  participants  (e.g.,  people 
with  a  learning  disability,  like  dyslexia,  or  a  physical  disability  which  prevents  them  from 
holding  books  and  turning  pages).  It  is  possible  that  these  factors  may  influence  the  degree  to 
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which  individuals  are  involved  in  their  own  health  care  decision-making,  the  amount  of  research 
"buy-in"  or  how  motivated  people  are  to  participate,  and  their  ability  to  successfully  navigate  the 
document.    There  may  also  be  implications  for  the  level  of  understanding  the  content, 
b)  "Usability"  research  -  bias  and  added-value.  Usability  testing,  conducted  by  an 
anthropologist  in  the  user's  home  environment  adds  an  ethnographic  perspective  and  the 
benefits  of  participant  observation.   In  this  case,  it  means  more  accurate  responses  to  how  the 
audio  handbook  is  actually  used,  i.e.,  whether  they  had  difficulties  opening  the  package,  or 
operating  the  tape  recorder.   Users  are  also  more  at  ease,  tending  to  make  their  comments 
more  forthcoming  and  meaningful  in  terms  of  their  health  care  information-seeking  and  amount 
of  decision-making.  Finally,  and  importantly,  this  technique  clearly  reduces  "respondent 
burden"  both  in  terms  of  their  not  having  to  travel,  and  comfort  during  the  testing.  This  makes 
it  possible  for  people  to  participate  who  might  not  agree  to  testing  because  of  the  travel  (i.e., 
elderly  and/or  newly  blind). 

Nonetheless,  bias  is  still  present  because  users  accommodate  the  researcher's  presence 
in  their  homes;  their  behavior  is  still  one  of  performance,  rather  than  actual  behavior  in  a 
natural  context. 
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CMS  FOCUS  GROUP  -  MODERATOR' S  GUIDE 

APPENDIX  A 

I.  Introduction 

A)  Thank  you 

B)  Confidentiality 

C)  Explain  process 

•  State  name  each  time  you  speak. 

•  One  word  answers  sometimes  get  lost. 

•  Jump  in,  we're  not  going  to  call  on  you  but  I  may  have  to  cut  you  off. 

•  All  consumers  who  are  very  familiar  with  reading  in  an  audio  format,  as 
consumers,  we  have  asked  you  all  to  participate  in  helping  us  find  ways 
to  make  the  handbook  clearer  and  easier  to  use. 

Several  parts  to  our  discussion  today:   First,  I'd  like  us  to  discuss  your  overall 
impression  of  the  audio  edition  of  the  Medicare  and  You  handbook.   Then,  we  will  move  on  to 
a  discussion  of  specific  portions  of  the  book,  with  a  focus  on  both  content  and  navigation.   As 
we  go  through  this  section,  I  am  going  to  ask  you  how  helpful  certain  elements  of  the  reference 
book  were.   And  lastly,  I'm  going  to  ask  you  to  discuss  how  easy  or  difficult  it  was  to 
accomplish  the  specific  tasks  we  assigned  you,  if  you  haven't  brought  that  up  already.   Please 
remember  we  are  testing  the  handbook,  not  your  ability  to  work  with  it.    Does  anyone  have 
any  questions? 

II.  General  Discussion 

I .  Icebreaker 

•  What  do  you  like  or  dislike  most  about  reading  by  audio  tape? 

II.  Transition 

•  What  about  for  reference  books? 
C)        Usefulness  Questions 

•  How  useful  was  this  (as  a  reference)  to  you? 

•  What  aspects/elements  were  particularly  helpful? 

•  Any  that  were  particularly  problematic?  Or,  not  that  helpful? 

•  What  recommendations  would  you  have  for  Medicare  to  make  it  more 
useful? 
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•  What  other  reference  materials  have  you  found  to  be  really  useful? 

•  Did  you  know  prior  to  us  contacting  you,  that  Medicare  had  things  in 
alternate  formats? 

•  Where  would  you  go  first  for  information? 

About  content/comprehension... 

Was  the  language  clear? 

How  was  the  narrator's  voice? 

How  was  the  speed  of  the  recording? 

Was  the  tone  audible  at  the  beginning  of  each  section? 

What  about  in  fast  forward  to  rewind  mode? 

Was  the  phone  number  clear  enough,  e.g.,  too  fast,  too  slow? 

IV.       Helpfulness  of  Sections 

How  helpful  was... 

•  the  introductory  material  (including  the  opening  statement),  table  of 
contents,  and  how  to  use  this  book 

•  the  mini  table  of  contents  read  at  the  beginning  of  sides  2  through  4 

•  the  question  and  answer  section 

•  Was  it  useful  for  navigating? 

•  Did  it  help  you  to  remember  the  upcoming  titles  with  corresponding 
tones? 

E)        Questions  re:  Individuals  Specific  Tasks 

•  For  those  of  you  who  used  them,  how  were  the  chart  descriptions?  At 
what  point,  if  at  all,  did  your  attention  fade?  How  useful  is  it  as  an 
audio  tool? 

•  For  those  of  you  with  this  assignment,  was  it  clear  what  was  meant  by 
"Medicare,  Part  B?" 


For  All: 


Was  there  anything  that  interrupted  the  flow  of  the  text,  that  you  found 
disconcerting  or  troublesome,  that  made  it  more  difficult  to  follow? 

How  did  you  feel  about  "Section  1/tone  3/pages  16-21"?  Was  this 
helpful  to  you?  Confusing?  Too  much  information  at  one  time?  Were 
you  able  to  index  things  this  way?  Could  you  remember  which  tones  to 
look  for?  Did  it  help  to  have  the  pages  read? 


Sidebar  information:  Would  you  like  to  hear  it  in  the  beginning  of  the 

section,  e.g.,  "This  is  going  to  be  15  pages,  discussing  topics 

instead  of  just  section  four,  tone  four? 
Did  it  help  to  have  the  page  numbers  read? 


III.  Navigation 

•  How  usable  did  you  find  this  handbook  to  be? 

•  Was  it  easy  to  navigate? 

•  What  parts  didn't  work  well? 

•  Are  the  index  and  table  of  contents  useful?  Which  was  used  more? 

According  to  your  specific  tasks... 


Were  you  able  to  find  what  we  asked  you  to  find? 

Were  the  telephone  listings  easy  to  find?  Were  you  able  to  find  where  to 

call  for  help? 

Were  you  able  to  find  the  question  and  answer  section? 

What  about  the  glossary  of  important  terms? 

How  did  you  go  about  doing  it? 

How  well  did  that  work  for  you?  Did  you  feel  it  was  pretty  easy  to  look 

that  up? 

About  how  long  did  it  take? 

Did  you  feel  it  was  worth  your  time? 

While  you  were  looking  up  information,  did  you  ever  think,  "I  would 

just  use  the  phone  number,"  or,  "I'd  prefer  to  have  someone  just  look  it 

up  for  me  in  the  regular  print  version,  it  would  be  easier."   Or  "I'll  just 

look  it  up  on  the  web? " 

Would  the  audio  cassette  be  the  first  place  you'd  go,  for  general 

information  or  if  you  needed  to  look  something  up? 

What  are  your  personal  preferences? 


IV. 


Conclusion 


Do  you  feel  it  met  your  basic  needs? 

Could  you  find  answers  in  a  reasonable  amount  of  time  and  with 

reasonable  accuracy? 

Was  there  anything  you  would  have  deleted  from  the  book? 

Was  there  anything  you  would  have  added? 

Do  you  have  any  advice  for  us  about  what  to  ask  less-experienced  audio 

readers? 


If  you  have  questions,  where  do  you  currently  get  your  information, 
e.g.,  insurance  company,  medicare,  social  security? 
What  kind  of  information  do  you  look  for,  e.g.  what's  covered,  what's 
not,  medications? 


CMS  "USABILITY"     MODERATORS  GUIDE 

APPENDIX  B 

I.  Introduction  [5  Minutes] 

A)  Thank  you 

B)  Confidentiality  /  Taping 

C)  Explain  process: 

•  "Thinking  aloud  method. . . " 

•  Background:  As  you  know,  we  are  interviewing  people  who  are 
Medicare  beneficiaries  of  various  backgrounds  to  see  how  well  the  audio 
version  of  the  handbook  works  for  you.   Please  keep  in  mind  as  we  go 
through  this  that  we  are  testing  the  handbook,  NOT  YOU.  Please  be 
honest  and  open. 

•  Purpose:   We  have  asked  you  to  participate  in  helping  us  find  ways  to 
make  the  handbook  clearer  and  easier  to  use.    Your  contribution  is 
invaluable  and  will  help  us  evaluate  the  tapes  as  a  reference  tool.   We  are 
going  to  focus  on  several  aspects  over  the  course  of  our  discussion. 
First,  we  will  simply  listen  to  the  tapes.   Then  I  will  begin  asking  you 
about  it.  I  am  going  to  want  to  hear  your  overall  impressions,  as  well  as 
focus  some  of  the  discussion  specifically  towards  issues  of  navigation 
and  content. 

•  Do  you  have  any  questions  for  me  at  this  point? 

II.  General  Discussion  [10  minutes] 

A)  Icebreaker 

•  How  do  you  generally  get  your  health  insurance  information? 

•  Do  you  prefer  materials  on  tape,  diskette,  braille,  (other)? 

•  What  do  you  like  or  dislike  most  about  reading  by  audio  tape? 

B)  Transition 

What  about  for  reference  books? 

Have  you  ever  used  audio  tapes  for  other  reference  books  before? 

What  other  things  have  you  listened  to  that  were  done  really  well? 

How  well  did  they  work? 

What  other  reference  materials  have  you  found  to  be  really  useful? 

Did  you  know  prior  to  us  contacting  you,  that  Medicare  had  things  in 


alternate  formats? 

When  you  receive  a  new  tape  such  as  this  one,  do  you  listen  all  the  way 
through  first,  or  do  you  jump  ahead  to  find  the  information  that  you  need? 
Do  you  tend  to  hear  a  woman's  voice  better  than  a  man's? 


C)  About  the  Packaging 

•  If  you  don't  read  braille,  and  you  dropped  the  tape,  how  would  you  know 
what  to  listen  to/for  if  the  tapes  got  mixed  up? 

•  How  easy  was  it  for  you  to  open  the  package? 

•  If  you  don't  read  braille  and  this  package  arrived  in  the  mail,  how  would 
you  know  what  it  was? 

III.  Task  1:   Open  and  Listen  to  Beginning  of  Tape 

What  are  your  initial  impressions? 

Was  the  language  clear? 

How  was  the  narrator's  voice? 

Speed  of  the  recording? 

Was  the  tone  audible  at  the  beginning  of  each  section? 

Was  the  phone  number  clear  enough?   (too  fast?  too  slow?) 

Do  you  know  what  tone  indexing  is? 


IV.  Task  2:   Navigation 

A)  Navigational  Task 

Now  that  you've  had  the  chance  to  listen  to  the  introduction  and  a  little  of  the 
content,  I'd  like  to  ask  you  to  look  up  something. 

•  What  kinds  of  things  do  you  look  for  when  trying  to  obtain  health 
insurance  information,  e.g.  what's  covered,  what's  not,  medications? 

•  Is  there  something  you  don't  know  about  Medicare  that  you've  always 
wanted  to  know? 

•  Is  there  an  issue  in  your  family,  or  something  that  we  can  look  up  right 
now  and  help  you  find  the  answer  to? 

•  How  was  that  to  work  with? 

•  How  were  you  able  to  find  that  section? 

•  What  would  have  made  it  easier? 

B)  Helpfulness  of  Sections 


How  help  was... 


the  introductory  material,  including  the  opening  statement;  table  of 

contents;  and  how  to  use  this  book? 

the  mini  table  of  contents  read  at  the  beginning  of  each  side?  Did  it  help 

you  to  remember  the  upcoming  titles  with  corresponding  tones? 

the  index  and  table  of  contents? 


C)  Ease  of  Navigation 


Would  you  consider  the  handbook  easy  to  navigate? 

What  parts  didn't  work  well? 

Was  there  anything  that  interrupted  the  flow  of  the  text,  that  you  found 

disconcerting  or  troublesome,  that  made  it  more  difficult  to  follow? 

How  did  you  feel  about  "Section  1/tone  3/pages  16-21",  e.g.  was  this 

helpful  to  you?  Confusing?  Too  much  information  at  one  time?  Were 

you  able  to  index  things  this  way?  Could  you  remember  which  tones  to 

look  for?  Did  it  help  to  have  the  pages  read? 

Sidebar  information:  Would  you  like  to  hear  it  in  the  beginning  of  the 

section,  e.g.,  "This  is  going  to  be  15  pages,  discussing  topics ," 

instead  of  just  section  four,  tone  four."   Did  it  help  to  have  the  page 

numbers  read? 

Do  you  feel  that  this  would  have  been  a  productive  use  of  your  time,  or 

would  you  have  given  up  and  used  the  phone  to  call  for  this  information? 

Would  you  have  thought  to  look  it  up  on  the  web? 

If  you  would  have  preferred  to  call  the  (800)  number,  or  if  you  wanted  to 

order  this  by  calling  1-800-..,  how  does  it  compare?  e.g.,  when  you  call  a 

number  and  press  one  for  this,  two  for  that  (phone  trees). 


V.  Task  3:   Content  Evaluation  [15  minutes  -  5  for  listening] 

Now  that  I  have  had  you  locate  specific  sections,  let's  listen  to  a  sample  that  I've 
already  cued  up  and  is  ready  to  go. 

•  Would  you  care  to  listen  to  that  again? 

•  What  is  meant  by  "Medicare  Part  B"? 


VI.  Conclusion  /  Overall  Impressions  [15minutes] 

•  How  useful  was  this  (as  a  reference)  to  you? 


What  aspects/elements  were  particularly  helpful? 

Any  that  were  particularly  problematic?  Or,  not  that  helpful? 

What  recommendations  would  you  have  for  Medicare  to  make  it  more 

useful? 

Do  you  feel  that,  as  it  is  now,  it  meets  your  basic  needs  as  a  reference 

document? 

Was  there  anything  you  didn't  want  in  the  book?  Or,  anything  you  think 

we  should  add? 

Now  that  you've  listened  once,  would  you  listen  again?  Would  you  use 

this  format  again? 

If  you  hadn't  listened  to  an  audio  cassette,  would  you  listen  all  the  way 

through? 

Do  you  have  any  questions  for  me? 

Would  it  be  ok  to  contact  you  again  by  phone,  and  follow  up  on  what 

you've  told  us  today? 


Thanks  again.  We  will  send  you  a  copy  of  the  report.  We  will  also  send  copy  of  the  revised 
audio  edition  next  year. 


April  23,  2001 

APPENDIX  C 

Health  Care  Financing  Administration  (HCFA) 
Technical  Direction  Letter  (T.D.L.) 
Task  Order  for: 

The  American  Foundation  for  the  Blind  (AFB) 

Corinne  Kirchner,  Director  of  Policy  Research  &  Program 

Evaluation 

11  Penn  Plaza,  Suite  300 

New  York,  NY  10001 

Consumer  Research:  Medicare  &  You  2001  Handbook,  Audio  Edition,  Revised 

November  2000 


Project  Description: 

Conduct  consumer  research  for  the  audio  version  of  the  Medicare  &  Yon  2001  handbook.  CMS 

publication  10050RE,  (English  edition)  revised  November  2000.  The  basis  for  the  M&Y 

consumer  research  is  to: 

—  Determine  what  navigation  patterns  consumers  follow  when  reviewing  the  recorded  edition  of 

the  handbook. 

~  Assess  the  usefulness  of  Medicare  health  insurance  information  in  an  audio  format. 

~  Evaluate  current  audio  format/presentation  of  the  handbook  and  recommend  changes. 


Background: 

Since  1978.  the  Health  Care  Financing  Administration  (now,  CMS)  has  produced  an  audio 
edition  of  the  Medicare  handbook.  On  average,  the  annual  total  of  recorded  handbooks 
requested,  from  1978  through  1998,  never  exceeded  1,500.  Thanks  to  timely  production  and 
vigorous  product  promotion  by  CMS's  Center  for  Beneficiary  Services  and  the  various 
Medicare  outreach  partners,  production  levels  for  the  recorded  edition  of  the  handbook  soared  to 
15.000  in  1999.  For  the  years  2000/2001,  requests  for  the  recorded  handbook  are  anticipated  to 
reach  an  all  time  high  of  30,000.  The  decision  to  consumer  test  the  audio  edition  of  the 
Medicare  handbook  comes  as  the  direct  result  of  the  product's  growing  popularity,  coupled 
with  CMS's  commitment  to  provide  blind,  low  vision  and  print  disabled  consumers  with  equal 
and  timely  access  to  high  quality  information  about  Medicare. 


Recruitment  Parameters: 

A.)  This  research  will  include  one  focus  group  session  with  5-8  individuals,  and  in-person 
interview/observation  sessions  with  8-10  blind,  low  vision,  and  print  disabled  individuals,  who 
fit  one  of  the  following  category  parameters: 
~  Medicare-entitled  beneficiaries,  ages  65  to  approximately  75; 

—  Disabled  beneficiaries,  entitled  to  Medicare  for  at  least  12  months,  ages  21  up  to  65; 

--  Blind  or  low  vision  individuals  who  provide  care  or  health  care  advice  for  Medicare  entitled 

beneficiaries; 

~  Members  of  national  advocacy  groups  for  the  Blind  (e.g.,  NFB,  ACB)  or  others  with  strong 

audio-listening  skills,  who  may  or  may  not  have  a  direct  relationship  with  Medicare  entitled 

beneficiaries 

B.)  Participants  in  the  consumer  testing  of  the  recorded  Medicare  handbook  must: 

~  Be  certifiable  as  blind,  low  vision  or  reading  disabled 

~  Have  a  minimum  of  12  months"  experience  using  recorded  materials  such  as  books  or 

periodicals  provided  by  the  National  Library  Service;  recorded  textbooks,  reports  or  newsletters. 

—  Represent  major  U.S.  regions; 

—  If  entitled  to  Medicare,  have  some  decision-making  responsibility  for  personal  care  matters; 
--  Not  be  a  resident  of  a  nursing  home; 

—  Not  be  enrolled  in  a  managed  care  plan; 

—  Agree  to  listen  to  the  audio  recording  of  the  2001  Medicare  handbook,  in  its  entirety  if 
possible,  but  at  least  to  cover  pre-determined  sections,  prior  to  the  telephone  interview  session; 
~  Agree  to  be  available  for  either  a  telephone  or  in-person  interview  session 

~  Not  be  dually  entitled  to  Medicare  and  Medicaid 


Consumer  Testing  the  Audio  Edition  of  the  Medicare  &  You  2001  Handbook 

Consumer  testing  of  the  recorded  edition  of  the  handbook  will  be  primarily  conducted  by  way  of 
in-person  interview/observation  sessions  and  a  telephone  focus  group.  The  general  approach  is 
"usability  research"  in  which  observation  of  how  materials  are  used  is  central.  The  overall  group 
of  participants  will  reflect  reasonable  gender  balance;  geographic  diversity  will  be  sought  for  the 
focus  group.  Data  collection  will  be  scheduled  over  a  period  of  two-  three  weeks;  all  sessions 
will  be  audio-recorded.  For  the  convenience  of  all  parties  concerned,  and  in  the  interest  of 
budget  constraints,  the  duration  of  each  session  will  not  exceed  2.0  hours. 


Communications  Analysis: 

Perform  a  communications  analysis  of  the  recorded  edition  of  the  M&Y  2001  handbook  to 
evaluate  current  product  features  such  as,  descriptions  of  charts  and  graphics;  structure  of 
recorded  handbook  elements;  tone  and  page  references.  Make  recommendations  from  a 
communications  perspective  that  illustrates  how  improvements  to  the  recorded  handbook  could 
enhance  consumer  use  and  understanding  of  Medicare  health  insurance  information  concepts. 


Research  Project  Questions: 

1 .  Does  the  recorded  edition  of  the  M&Y  2001  handbook  meet  basic  information  needs  of  the 
target  audience? 

2.  Do  recorded  publications  need  a  different  style  of  information  organization  from  that  of  the 
print  edition  in  order  to  be  used  as  an  effective  reference  tool? 

3.  Is  the  level  of  detail  included  in  the  audio  production  of  the  handbook,  pagination; 
descriptions  of  charts  and  graph;  phone  number  listings,  adequate? 

4.  Can  the  users  of  the  recorded  handbook  find  answers  to  their  Medicare  questions  in  a 
reasonable  time  and  with  good  accuracy? 

5.  How  do  users  of  the  audio  edition  of  the  M&Y  2001  handbook  react  to  the  type  and  voice 
quality  of  the  reader  (gender,  tone,  etc.)  in  terms  of  satisfaction  and  information  needs? 

Deliverables: 

Overall  research  plan  and  costs. 

Screening  instrument,  moderator's  guide,  interview/observation  guide 

Topline  report  and  presentation  of  research  findings 

Final  report  on  consumer  research 
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